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TITLE OF THE INVENTION 
COLOR REPRODUCTION SYSTEM 

CROSS-REFERENCE TO RELATED APPLICATIONS 
This application is based upon and claims the 
benefit of priority from the prior Japanese Patent 
Application No. 11-144932, filed May 25, 1999, the 
entire contents of which are incorporated herein by 
reference. 

BACKGROUND OF THE INVENTION 
The present invention relates to a reproduction 
system having a color image processing apparatus for 
calculating tristimulus values under observation 
illumination light corresponding to the spectral 
reflectance of an object. 

In recent years, color management systems (CMS) 
for matching the colors of input/output images between 
a plurality of color image devices such as a color 
CRT monitor and color printer are becoming popular in 
various fields handling color images. 

In a CMS, a problem is arisen in accurately 
reproducing colors to be perceived when sensing 
illumination light in sensing an object is different 
from observation illumination light in observing 
an image obtained by displaying image data obtained by 
sensing on a CRT monitor or outputting it by a printer 
because the colors of the observed image are different 
between these environments due to the change in human 
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visual characteristics. 

As a solution to this problem, a color 
reproduction method is employed in which the spectral 
reflectance f(X) of the object is obtained, and CIE XYZ 
5 values of the reflectance spectrum are calculated by 

multiplying the spectral reflectance by the spectrum 
Eo( X ) of observation illumination light for display . 

The spectral reflectance f(X) is obtained in 
correspondence with each pixel of the object image 
10 obtained by sensing the object. The CIE XYZ values are 

calculated by equations (1) using XYZ color matching 
functions x(l), y{X), and z(X) defined by the 
International Commission on Illumination (CIE). 
780 

X = I yL(X)E 0 (X)f(X)dX 
\ = 380 

780 

15 Y - J y(X)E o a)f(\)di\ 

X = 380 

780 

z = jz(^)E 0 a)fa)d^ 

A, = 380 

where X is the wavelength [nm]. In this method of 
obtaining the spectral reflectance f(X) corresponding 
to each pixel of the object image and calculating the 
XYZ values under the observation illumination light 
20 on the basis of the spectral reflectance for display, 

the XYZ values when the object is present under 
the observation illumination light are displayed. 
For this reason, accurate colors to be perceived can be 
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reproduced without using a visual model for chromatic 
adaptation. 

In the above-described conventional color 
reproduction system, to calculate the XYZ values under 
5 the observation illumination light from the spectral 

reflectance f ( X ) of the object, the spectrum Eo( X ) of 
the observation illumination light must be obtained . 

However, to accurately measure the spectrum, a 
measurement instrument such as a spectrophotometer is 

10 necessary. Since spectrophotometers must be prepared 

at various observation positions to measure the 
spectrum of the observation illumination light Eo(A), 
resulting in cumber someness . 

BRIEF SUMMARY OF THE INVENTION 

15 It is an object of the present invention to 

provide a color reproduction system having a color 
image processing apparatus for calculating tristimulus 
values under observation illumination light corre- 
sponding to the spectral reflectance of an object 

2 0 without using any measurement instrument such as 

a spectrophotometer, unlike the prior art. 

In order to solve the above problem, as a 
characteristic feature of the present invention, in a 
color reproduction system comprising tristimulus value 

25 calculation means for calculating tristimulus values 

under observation illumination light corresponding to 
a spectral reflectance of an object, color image signal 
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calculation means for calculating an output color image 
signal on the basis of the tristimulus values, and 
color image output means for outputting a color image 
upon receiving the output color image signal , the 
5 tristimulus value calculation means calculates the 

tristimulus values using spectral reflectance data of 
a color chip formed from a plurality of unit color 
chips, color chip sensing data obtained by sensing the 
color chip with an input device under the observation 

10 illumination light, spectral sensitivity data of the 

input device, and color matching function data. 

In the present invention having this arrangement, 
the tristimulus values under the observation illumina- 
tion light corresponding to the spectral reflectance of 

15 the object can be obtained using a color chip having 

a known spectral reflectance and an input device for 
sensing the color chip, which has a known spectral 
sensitivity. That is, the spectral reflectance 
image data can be converted into XYZ image data as 

2 0 tristimulus values under the observation illumination 

light without using any measurement instrument such as 
a spectrophotometer . 

As a characteristic feature of an aspect, the 
tristimulus value calculation means multiplies the 

25 color chip sensing data by a matrix obtained from 

the relationship between the product of the spectral 
reflectance of the object and the color matching 
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functions and the product of the spectral reflectance 
of the color chip and the spectral sensitivity of the 
input device, thereby calculating the tristimulus 
values • 

5 When the product of the spectral reflectance of 

the object and the color matching functions can be 
expanded by the product of the spectral reflectance of 
the color chip and the spectral sensitivity of the 
input device for sensing the color chip, spectral 

10 reflectance image data can be highly accurately 

converted into XYZ image data as the tristimulus values 
under the observation illumination light. 

Even when such expansion is impossible, highly 
accurate conversion can be performed by increasing the 

15 expansion accuracy by changing the color chip to be 

used and the characteristics of the input device for 
sensing the color chip in accordance with the spectral 
reflectance of the object. 

As a characteristic feature of another aspect, 

2 0 the tristimulus value calculation means obtains the 

tristimulus values as the linear sum of basis function 
tristimulus values obtained by multiplying the color 
chip sensing data by a matrix obtained from the 
relationship between the product of the plurality of 

25 basis functions of the spectral reflectance of the 

object and the color matching functions and the product 
of the spectral reflectance of the color chip and the 
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spectral sensitivity of the input device. 

When the product of the plurality of basis 
functions of the spectral reflectance of the object and 
the color matching functions can be expanded by the 
5 product of the spectral reflectance of the color chip 

and the spectral sensitivity of the input device for 
sensing the color chip, spectral reflectance image data 
can be highly accurately converted into XYZ image data 
as the tristimulus values under the observation 

10 illumination light. 

Even when such expansion is impossible, highly 
accurate conversion can be performed by increasing the 
expansion accuracy by changing the color chip to be 
used and the characteristics of the input device for 

15 sensing the color chip in accordance with the basis 

function of the spectral reflectance of the object. 

When all spectral reflectances of the object 
are expanded by the basis functions, the conversion 
accuracy for the actual object can be obtained in 

2 0 advance by evaluating the conversion accuracy for 

the basis function in advance. 

In addition, the tristimulus values under the 
observation illumination light corresponding to the 
basis function of the spectral reflectance of the 

25 object can be calculated in advance. For this reason, 

after the expansion coefficient corresponding to the 
spectral reflectance of the object is input, the 



7 - 



tristimulus values under the observation illumination 
light corresponding to the spectral reflectance of the 
object can be calculated at a high speed as the linear 
sum of the tristimulus values of the basis functions. 
5 More specifically, according to the first aspect 

of the present invention, there is provided a color 
reproduction system comprising: 

tristimulus value calculation means for calculat- 
ing tristimulus values under observation illumination 
10 light corresponding to a spectral reflectance of 

an object, the tristimulus value calculation means 
calculating the tristimulus values using spectral 
reflectance data of a color chip formed from a 
plurality of unit color chips, color chip sensing data 
15 obtained by sensing the color chip with an input device 

under the observation illumination light, spectral 
sensitivity data of the input device, and color 
matching function data; 

means for calculating an output color image signal 
20 on the basis of the calculated tristimulus values; and 

means for outputting a color image on the basis of 
the output color image signal. 

According to the second aspect of the present 
invention, there is provided a color reproduction 
25 system comprising: 

first image sensing means for sensing an object 
under sensing illumination light; 
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means for calculating a spectral reflectance of 
image data of the object sensed by the first image 
sensing means on the basis of spectral sensitivity data 
of the first image sensing means, spectrum data of the 
5 sensing illumination light , statistic data of a spectra 

reflectance of the object , and color matching function 
data, and outputting spectral reflectance image data 
corresponding to the calculated spectral reflectance; 

second image sensing means for sensing a color 
10 chip under observation illumination light; 

means for calculating tristimulus values of the 
object under the observation illumination light on the 
basis of the output spectral reflectance image data, 
color chip image data of the color chip sensed by the 
15 second image sensing means, spectral sensitivity data 

of the second image sensing means, color chip spectral 
reflectance data representing a spectral reflectance of 
the color chip, and color matching function data; 

means for calculating an output color image signal 
20 on the basis of the calculated tristimulus values; and 

means for outputting a color image on the basis of 
the output color image signal. 

According to the third aspect of the present 
invention, there is provided a color reproduction 
25 system comprising: 

first image sensing means for sensing an object 
under sensing illumination light; 
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means for outputting expansion coefficient data 
representing a linear sum of basis functions of a 
spectral reflectance of the object on the basis of 
spectral sensitivity data of the first image sensing 
5 means, spectrum data of the sensing illumination light, 

statistic data of a spectral reflectance of the object, 
and color matching function data; 

second image sensing means for sensing a color 
chip under observation illumination light; 

10 means for calculating tristimulus values of the 

object under the observation illumination light on the 
basis of the output expansion coefficient data, color 
chip image data of the color chip sensed by the second 
image sensing means, spectral sensitivity data of 

15 the second image sensing means, color chip spectral 

reflectance data representing a spectral reflectance 
of the color chip, and color matching function data; 

means for calculating an output color image signal 
on the basis of the calculated tristimulus values; and 

20 means for outputting a color image on the basis of 

the output color image signal. 

Additional objects and advantages of the invention 
will be set forth in the description which follows, and 
in part will be obvious from the description, or may 

2 5 be learned by practice of the invention. The objects 

and advantages of the invention may be realized and 
obtained by means of the instrumentalities and 
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combinations particularly pointed out hereinafter. 

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING 
The accompanying drawings, which are incorporated 
in and constitute a part of the specification, illust- 
5 rate presently preferred embodiments of the invention, 

and together with the general description given above 
and the detailed description of the preferred embodi- 
ments given below, serve to explain the principles of 
the invention. 

10 FIG. 1 is a block diagram showing the arrangement 



of a color reproduction system according to the first 
embodiment of the present invention; 

FIG. 2 is a block diagram showing the arrangement 
of a spectral reflectance calculation unit of the first 



15 



embodiment; 



FIG. 3 is a block diagram showing the arrangement 



of a tristimulus value calculation unit of the first 



embodiment ; 



FIG. 4 is a block diagram showing the arrangement 



20 



of a color image signal calculation unit of the first 



embodiment ; 



FIG. 5 is a block diagram showing the arrangement 



of a color reproduction system according to the second 



embodiment of the present invention; 



25 



FIG. 6 is a block diagram showing the arrangement 



of a spectral reflectance calculation unit of the 



second embodiment ; and 
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FIG. 7 is a block diagram showing the arrangement 
of a tristimulus value calculation unit of the second 
embodiment . 

DETAILED DESCRIPTION OF THE INVENTION 
5 The embodiments of the present invention will be 

described below with reference to the accompanying 
drawing, 

(First Embodiment) 

FIG. 1 shows a color reproduction system according 
10 to the first embodiment of the present invention. 

When an object 1 is sensed by a first digital 
camera 3 under sensing illumination light by sensing 
illumination 2, an object image signal is output from 
the digital camera 3 as RGB image data. This RGB image 
15 data is input to a spectral reflectance calculation 

unit 4. 

Upon receiving the RGB image data from the digital 
camera 3, the spectral reflectance calculation unit 4 
calculates, as a spectral reflectance f(^l) correspond- 

2 0 ing to each pixel of the RGB image data, the spectral 

reflectance f(X) of the object 1 using the spectral 
sensitivity data of the digital camera 3, spectrum data 
of the sensing illumination light, statistic data of 
the object spectral reflectance, and color matching 

2 5 function data. 

Image data (called spectral reflectance image 
data) F corresponding to the calculated distribution of 
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the spectral reflectance f ( X ) is output. The spectral 
reflectance image data F is input to a tristimulus 
value calculation unit 5, 

When a color chip 6 is sensed by a second digital 
5 camera 8 as an input device under observation illumina- 

tion light by observation illumination 7, color chip 
image data P 1 corresponding to the color chip 6 is 
output from the digital camera 8. 

The color chip 6 is formed by arraying a plurality 

10 of (M) unit color chips having independent spectral 

reflectances in a matrix . The color chip 6 is laid out 
without any illumination variation and sensed by the 
digital camera 8. The color chip image data P 1 is 
input to the tristimulus value calculation unit 5, 

15 The tristimulus value calculation unit 5 

calculates tristimulus values X, Y, and Z corresponding 
to the spectral reflectance of the object 1 under 
the observation illumination light from the spectral 
reflectance image data F input from the spectral 

20 reflectance calculation unit 4, the color chip image 

data P' input from the second digital camera 8, the 
spectral sensitivity data of the digital camera 8, the 
color chip spectral reflectance data representing the 
spectral reflectance distribution of the color chip 6, 

25 and color matching function data, and outputs XYZ image 

data. 

The XYZ image data are input to a color image 
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signal calculation unit 9. The spectral sensitivity 
of the digital camera 8, the color chip spectral 
reflectance, and color matching functions have known 
values and stored and held in the tristimulus value 
5 calculation unit 5 in advance. 

The color image signal calculation unit 9 converts 
the XYZ image data input from the tristimulus value 
calculation unit 5 into output color image signals R* , 
G 1 , and B 1 using a monitor profile as input/output 
10 characteristic information of a CRT monitor 10, which 

is obtained in advance by measurement. 

The output color image signals R' , G', and B* are 
input to the CRT monitor 10 as a color image output 
means whereby an object image having the colors of the 
15 object 1 under the observation illumination light is 

displayed on the CRT monitor 10. 

The detailed arrangement and processing of each 
section shown in FIG. 1 will be described next in 
detail . 

20 FIG. 2 shows the arrangement of the spectral 

reflectance calculation unit 4. 

The spectral reflectance calculation unit 4 

comprises a matrix conversion unit 21, storage unit 22, 

and matrix calculation unit 23. 
2 5 The RGB image data from the first digital camera 3 

is input to the matrix conversion unit 21. The matrix 

conversion unit 21 outputs the spectral reflectance 
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image data F by multiplying the RGB image data by 
a matrix calculated by the matrix calculation unit 23 
on the basis of data stored in the storage unit 22. 
The spectral reflectance calculation unit 4 
5 shown in FIG. 2 will be described in more detail. 

The storage unit 22 stores and holds spectral 
sensitivity data H of the first digital camera 3, 
sensing illumination light spectrum data Em, and 
correlation matrix data A of the spectral reflectance 

10 of the object 1 as data at a 1-nm interval in the 

wavelength range of 3 80 to 780 nm. 

The sensing illumination light spectrum data Em is 
measured using a spectrophotometer or set in advance. 
The correlation matrix data A of the spectral 

15 reflectance of the object 1 is also calculated from 

data measured in advance. 

The matrix calculation unit 23 reads out the data 
H, A, and Em from the storage unit 22 and calculates a 
spectral reflectance estimation matrix M for converting 

2 0 the RGB image data from the digital camera 3 into the 

spectral reflectance image data F. 

The spectral reflectance estimation matrix M is a 
401 X 3 matrix which is used to convert the RGB image 
data as three-dimensional data into a 401-dimensional 

25 spectral reflectance at the 1-nm interval in the 

wavelength range of 380 to 780 nm. Let G be the RGB 
image data output from the digital camera 3 . 
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Error = ((F - MG) t (F - MG)) 



= {(F - MHE m F) (F - MHE m F) 



for 



H - 



r(380) , r(381) , , r(780) 

g(380) , g(381) , , g(780) 

b(380) , b(381) , , b(780) 

fE m (380) ,0, ,0^ 

0,E m (381) , ,0 



Em = 



F = 



1,0, 0, ,E m (780) J 

(f(380) , f(381) , f f(780)) fc 



M = 



m ll m 12 m 13 



m 21 m 22 m 23 



^m 40 il 1*4012 m 4013y 
M that minimizes the above matrix is given by 

M = A(HEmY(HEmA(HEmyy } 
for 

A = (FF f ) 

10 The matrix conversion unit 21 multiplies the RGB 

image data G by the spectral reflectance estimation 
matrix M and outputs the spectral reflectance image 
data F. 

FIG. 3 shows the arrangement of the tristimulus 
15 value calculation unit 5. 
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The tristimulus value calculation unit 5 comprises 
a tristimulus value matrix calculation unit 31 , storage 
unit 32 , image extraction unit 33, and tristimulus 
value matrix conversion unit 34. 
5 The spectral reflectance image data F output from 

the spectral reflectance calculation unit 4 shown in 
FIG. 1 or 2 is input to the tristimulus value matrix 
calculation unit 31. A tristimulus value matrix Mqc is 
calculated on the basis of data stored in the storage 

10 unit 32. 

The tristimulus value matrix Mqc is input to the 
tristimulus value matrix conversion unit 34. The 
tristimulus value matrix conversion unit 34 multiplies 
the tristimulus value matrix Mqc by color chip sensing 

15 data P extracted by the image extraction unit 33 to 

perform matrix conversion, thereby obtaining 
tristimulus values C. 

The tristimulus value calculation unit 5 shown in 
FIG. 3 will be described in more detail. The storage 

20 unit 32 stores and holds spectral sensitivity data H2 

of the second digital camera 8, color matching function 
data c, and spectral reflectance data g of the color 
chip 6 as data at a 1-nm interval in the wavelength 
range of 380 to 780 nm. 

25 The tristimulus value matrix calculation unit 31 

reads out the data H2, c, and g from the storage unit 
32 and calculates the tristimulus value estimation 
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matrix Mqc for converting the color chip sensing data 
input from the second digital camera 8 through the 
image extraction unit 33 into the tristimulus values C 
under the observation illumination light spectrum of 
5 the spectral reflectance image data F read from the 

spectral reflectance calculation unit 4. 

The image extraction unit 33 extracts data in each 
unit color chip region from the color chip image data 
P' obtained by sensing the color chip 6 with the second 

10 digital camera 8 and calculates the average signal 

value thereof to obtain the color chip sensing data P . 

The color chip 6 is constructed by a plurality 
of (M) unit color chips having independent spectral 
reflectances. Hence, 3M color chip sensing data P 

15 are obtained by sensing the M unit color chips with 

the digital camera 8 using three RGB channels and 
extracting the data by the image extraction unit 33. 

The tristimulus value estimation matrix Mqc is a 
3 X 3M matrix which is used to convert the 3M color 

20 chip sensing data P obtained by the three RGB channels 

into the tristimulus values C as three-dimensional XYZ 
image data. 

The tristimulus value estimation matrix Mqc 
is calculated in the following manner. Let hi( 2 )(A) 
25 (i = 1 to 3) be the spectral sensitivity data H2 of the 

second digital camera 8, q j ( X ) (j = 1 to M) be the 
spectral reflectance data q of the color chip 6, 
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and ci(X) (i = 1 to 3) be the color matching function 
data c. 

f[(K) = c± a)fa) a = i~3) 

q'Md-D+j <*<> = hfhxnjO,) a = i~3, j = i~m> 

fi = (fi^i) , fla 2 > / / f I<^N> > 

q* = <qi> <32/ — / q3M) fc 

qi = (qi(^i) , qi(A2> ,..../ qi(^N> > 
5 Mqc that minimizes 

E qc = (f'-Mqcq'^tf'-Mqcq 1 ) 
is calculated by 

10 M qc - f'q' fc (q f q ft ) _1 

More specifically, the tristimulus value 
estimation matrix Mqc is obtained from the relationship 
between the product of the spectral reflectance f(X) of 
the object 1 and the color matching functions ci(X) and 

15 the product of the spectral reflectance q j ( X ) of the 

color chip 6 and the spectral sensitivity hi( 2 )(/L) of 
the digital camera 8 as an input device for sensing the 
color chip 6, 

The calculated tristimulus value estimation matrix 

2 0 Mqc is input to the tristimulus value matrix conversion 

unit 34, so the tristimulus values C are calculated by 
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C = M qc P 

P = (Pi/ P2/ / P3M> fc 

C - (X, Y, Z) fc 

As is apparent from equations (8), when Eqc is 0, 
accurate tristimulus values C can be obtained by the 
5 above equations for an arbitrary observation illumina- 

tion light spectrum Eo. The tristimulus values C 
calculated by the tristimulus value matrix conversion 
unit 34 are input to the color image signal calculation 
unit 9 shown in FIG* 1. 

10 FIG • 4 shows the arrangement of the color image 

signal calculation unit 9. 

The color image signal calculation unit 9 
comprises an output RGB matrix conversion unit 41, 
storage unit 42, and gray scale correction unit 43. 

15 The tristimulus values C output from the 

tristimulus value matrix conversion unit 34 in the 
tristimulus value calculation unit 5 shown in FIG. 3 
are input to the color image signal calculation unit 9, 
In the output RGB matrix conversion unit 41, the 

2 0 tristimulus values C are converted into the RGB data 

G by equations (9) in accordance with an output RGB 
matrix Mem given by the storage unit 42. 
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G=M cm C 
for 

G = (R,G,B) 



M = 



7 7 7 



where Xr, Xg, and Xb are X values for R, G, and B 
maximum light emission. This also applies to Y and 
5 Z values. 

The RGB data G calculated by the output RGB matrix 
conversion unit 41 are input to the gray scale 
correction unit 43, and nonlinear gray scale correction 
is performed in accordance with gray scale correction 

10 data given by the storage unit 42, 

RGB data G f (corresponding to R' , G 1 , and B ? in 
FIG. 1) after gray scale correction are input to the 
CRT monitor 10 as output color image signals, and the 
color image of the object 1 is displayed on the CRT 

15 monitor 10. 

As described above, according to the first 
embodiment, by using the color chip 6 having a known 
spectral reflectance and the digital camera 8 as an 
input device for sensing the color chip 6, which has 

20 a known spectral sensitivity, the spectral reflectance 

image data F can be converted into XYZ image data 
as the tristimulus values C under the observation 
illumination light by the tristimulus value calculation 
unit 5 without using any measurement instrument such as 



a spectrophotometer . 

If the product of the spectral reflectance f ( X ) of 
the object 1 and the color matching functions ci(X) can 
be expanded by the product of the spectral reflectance 
5 q j ( X ) of the color chip 6 and the spectral sensitivity 

hx( 2 )(X) of the digital camera 8 as an input device 
for sensing the color chip 6, the spectral reflectance 
image data F can be highly accurately converted into 
the XYZ image data as the tristimulus values C under 

10 the observation illumination light. 

In this embodiment, more accurate conversion is 
enabled by changing the color chip 6 to be used and the 
characteristics of the second digital camera 8 as an 
input device for sensing the color chip 6 in accordance 

15 with the spectral reflectance f ( X ) of the object 1. 

( Second Embodiment ) 

FIG. 5 shows a color reproduction system according 
to the second embodiment of the present invention. 

In this embodiment, the output from a spectral 

20 reflectance calculation unit 14 becomes the expansion 

coefficient when the spectral reflectance of an object 
1 is represented as the linear sum of basis functions. 
The arrangement of this embodiment is the same as that 
described in the first embodiment except the spectral 

25 reflectance calculation unit 14 and tristimulus value 

calculation unit 15, and a detailed description thereof 
will be omitted. 
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The spectral reflectance calculation unit 14 
receives RGB image data from a digital camera 3 and 
outputs expansion coefficient data K when the spectral 
reflectance of the object 1 is represented by the 
5 linear sum of basis functions by using the spectral 

sensitivity data of the digital camera 3, spectrum data 
of sensing illumination light, and statistic data of 
the spectral reflection of the object 1. 

The expansion coefficient data K is input to the 

10 tristimulus value calculation unit 15. The tristimulus 

value calculation unit 15 also receives color chip 
image data P' corresponding to a color chip 6, which 
is output from a second digital camera 8 as an input 
device which senses the color chip 6 formed from 

15 a plurality of unit color chips under observation 

illumination light by observation illumination 7. 

On the basis of the expansion coefficient data K 
input from the spectral reflectance calculation unit 
14, the color chip image data P' output from the second 

20 digital camera 8, the spectral sensitivity data of the 

digital camera 8, the spectral reflectance data of the 
color chip 6, and the color matching function data, 
the tristimulus value calculation unit 15 calculates 
tristimulus values X, Y, and Z corresponding to the 

25 spectral reflectance of the object 1 under the 

observation illumination light and inputs the XYZ image 
data to a color image signal calculation unit 9* 



- 23 - 



The spectral sensitivity of the digital camera 8, 
the color chip spectral reflectance, and color matching 
functions have known values, and these data are stored 
and held in the tristimulus value calculation unit 15 
5 in advance, 

FIG. 6 shows the arrangement of the spectral 
reflectance calculation unit 14. The spectral 
reflectance calculation unit 14 comprises a matrix 
conversion unit 61, storage unit 62, and matrix 
10 calculation unit 63. 

The RGB image data from the first digital camera 3 
are input to the matrix conversion unit 61. The RGB 
data are multiplied by a matrix calculated by the 
matrix calculation unit 63 on the basis of data stored 
15 in the storage unit 62, so the expansion coefficient 

data K of the spectral reflectance is output. 

The storage unit 62 stores and holds spectral 
sensitivity data H of the first digital camera 3, 
correlation matrix data A ' of the expansion coefficient 
20 when the spectral reflectance of the object 1 is 

represented as the linear sum of basis functions, basis 
function data V of the spectral reflectance of the 
object 1, and sensing illumination light spectrum data 
Em as data at a 1-nm interval in the wavelength range 
25 of 380 to 780 nm. 

The matrix calculation unit 63 reads out the data 
H, A' , V, and Em from the storage unit 62 and 



calculates a expansion coefficient estimation matrix M 
for converting the RGB image data from the digital 
camera 3 into the expansion coefficient data K of the 
object spectral reflectance. 

The expansion coefficient estimation matrix M is 
an R X 3 matrix which is used to convert the 
three-dimensional RGB data into a expansion coefficient 
related to R basis functions. Let G be the RGB image 
data from the digital camera 3 . 



Error = ((K — MG) (K — MG) 




(K - MHE m F) (K - MHE m F) 




(K - MHE m VK) (K - MHE m VK) 



for 
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10 



H = 



^(380) , r(381) , , r(780) A 



g(380) , g(381) , , g(780) 

b(380) , b(381) , , b(780) 

fE m (380) , 0, ,0^ 

0, E m (381) , ,0 



Em 



0, 0, , E m (780) 



/ v 1 (380)v 2 (380) v R (380)^ 

vi(381)V2<381> v R (381) 



V = 



v 1 (780)v 2 (780) v R (780) 



K - (ki, k 2 , , k R r 



^ m ll m 12 m 13 ^ 



M - 



m 21 m 22 m 23 



\_ m Rl m R2 m R3j 
M that minimizes the above matrix is given by 



M = (K t K\(HEmV) t (HEmV t (K t K\(HEmV) t ) 1 



The matrix conversion unit 61 multiplies the RGB 
image data G by the calculated expansion coefficient 
estimation matrix M, thereby outputting the expansion 
coefficient data K. 

FIG. 7 shows the arrangement of the tristimulus 
value calculation unit 15. The tristimulus value 
calculation unit 15 comprises an image extraction 
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unit 71, tristimulus value matrix calculation unit 72, 
storage unit 73 , and tristimulus value matrix 
conversion unit 74. 

The color chip image data P' obtained by sensing 
5 the color chip 6 with the digital camera 8 is input 

to the tristimulus value matrix calculation unit 72 
through the image extraction unit 71. A tristimulus 
value estimation matrix Mqc is calculated on the basis 
of data stored in the storage unit 73. 

10 The tristimulus value estimation matrix Mqc is 

input to the tristimulus value matrix conversion unit 
74. in the tristimulus value matrix conversion unit 
74, the tristimulus value estimation matrix Mqc is 
multiplied by the expansion coefficient data K of the 

15 spectral reflectance of the object 1, which is output 

from the spectral reflectance calculation unit 14 f 
thereby obtaining tristimulus values C. 

Each section shown in FIG. 7 will be described in 
more detail. The storage unit 73 stores and holds 

20 spectral sensitivity H2 of the second digital camera 8, 

color matching function data c, spectral reflectance 
data g of the color chip 6, and basis function data V 
of the object spectral reflectance as data at a 1-nm 
interval in the wavelength range of 380 to 780 nm. 

25 The tristimulus value matrix calculation unit 72 

reads out the data H2 , c, q, and V from the storage 
unit 73 and calculates the tristimulus value estimation 
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matrix Mqc for converting color chip sensing data P 
input from the second digital camera 8 through the 
image extraction unit 71 into the tristimulus values 
C under the observation illumination light of the 
5 spectral reflectance corresponding to the expansion 

coefficient of the expansion coefficient data K read 
from the spectral reflectance calculation unit 14. 

The image extraction unit 71 extracts data in each 
unit color chip region from the color chip image data 

10 P' obtained by sensing the color chip 6 with the second 

digital camera 8 and calculates the average signal 
value, thereby obtaining the color chip sensing data P. 

The color chip 6 is constructed by a plurality 
of (M) unit color chips having independent spectral 

15 reflectances. Hence, 3M color chip sensing data P are 

obtained by sensing the M unit color chips with the 
digital camera 8 using three RGB channels and 
extracting the data by the image extraction unit 71. 

The tristimulus value estimation matrix Mqc is a 

20 3 X R matrix which is used to convert the R expansion 

coefficient data K into the tristimulus values C as 
three-dimensional XYZ image data. The tristimulus 
value estimation matrix Mqc is calculated in the 
following manner. Let hi( 2 )(X) (i = 1 to 3) be the 

25 spectral sensitivity data H2 of the digital camera 8 as 

an input device, q j ( X ) (j = 1 to M) be the spectral 
reflectance data q of the color chip 6, ci( X ) (i = 1 
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to 3) be the color matching function data c, VI (1) and 
kl (1 = 1 to R) be the basis function V and expansion 
coefficient K of the spectral reflectance of the object 
1, respectively. 

5 

R 

f(X) = £ k l v l<^> 

1 = 1 

f'nil) = c i a)v 1 (A,)(i = 1~3, 1 = 1~R) 

q'M(i-i)+j M = i4 2) a)qj<>.) a = i-3, j = i~m> 

f = (f u , f '21 , f '31 ) fc 

T ' ' ' 

f il = (£x±0^\) f f21<^2> t * * * * t f 3l(^N> ) 
1 ' 1 t 

q 1 = (qi, • • • • / q3M) 

q± = (q'it^!) , qi(X, 2 ) ...... q±(^N> > 

M( 1 )qc that minimizes 

E (1) qc - (^1 ~ M 1 qc q' ^(f^ - M 1 qc q' ) 
10 is calculated by 

M (1) qc - fiq ft <q f q ft > _1 
The tristimulus values ct 1 ) of the basis functions are 
obtained by 

C< X > = M< 1 > qc P 

P = (Pi/ P2' / P3M> t 

c (l) = ^(D^D^Djt 

The tristimulus value estimation matrix Mqc is given by 

M qc = (CWC^ C< R >) 

20 More specifically, the tristimulus value 



estimation matrix Mqc is obtained as the linear sum of 
the tristimulus values c( 1 ) of the basis functions, 
which is obtained by multiplying the color chip sensing 
data P by a matrix M (i) qc obtained from the relation- 
ship between the product of a plurality of basis 
functions vl(/L) for the spectral reflectance f(l) of 
the object 1 and the color matching functions ci(X) and 
the product of the spectral reflectance q j ( X ) of the 
color chip 6 and the spectral sensitivity hi( 2 )(l) of 
the digital camera 8 as an input device for sensing the 
color chip 6. 

The calculated tristimulus value estimation matrix 
Mqc is input to the tristimulus value matrix conversion 
unit 74 and converted into the tristimulus values C by 

C - M qc K 

As is apparent from equation (18), when E( 1 )qc 
(1 = 1 to R) is 0, accurate tristimulus values C can 
be obtained by the above equations for an arbitrary 
observation illumination light spectrum Eo. 

The tristimulus values C calculated by the 
tristimulus value matrix conversion unit 74 are input 
to the color image signal calculation unit 9 shown in 
FIG. 5, converted into RGB data as output color image 
signals, and input to a CRT monitor 10, as in the first 
embodiment. The arrangement of the color image signal 
calculation unit 9 is the same as in the first 
embodiment, and a detailed description thereof will be 
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omitted . 

As described above, according to the second 
embodiment, by using the color chip 6 having a known 
spectral reflectance and the digital camera 8 as an 
5 input device for sensing the color chip 6, which has 

a known spectral sensitivity, the spectral reflectance 
image data F can be converted into XYZ image data 
as the tristimulus values C under the observation 
illumination light by the tristimulus value calculation 

10 unit 15 without using any measurement instrument such 

as a spectrophotometer, as in the first embodiment. 

If the product of the plurality of basis functions 
vl(l) of the spectral reflectance f ( X ) of the object 1 
and the color matching functions ci(X) can be expanded 

15 by the product of the spectral reflectance q j ( X ) of the 

color chip 6 and the spectral sensitivity hi( 2 )(X) of 
the digital camera 8 as an input device for sensing the 
color chip 6, the spectral reflectance image data F can 
be highly accurately converted into the XYZ image data 

20 as the tristimulus values C under the observation 

illumination light. 

In this embodiment, more accurate conversion is 
enabled by changing the color chip 6 to be used and the 
characteristics of the second digital camera 8 as an 

2 5 input device for sensing the color chip 6 in accordance 

with the basis functions V of the spectral reflectances 
f(X) of the object 1. 
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According to this embodiment, since all of the 
spectral reflectances f(^l) of the object 1 are expanded 
by the basis functions V, the conversion accuracy for 
the actual object 1 can be obtained in advance by 
5 evaluating the conversion accuracy for the basis 

function V in advance. 

In this embodiment, the tristimulus values c(i) 
under the observation illumination light corresponding 
to the basis function V of the spectral reflectance 

10 f(^) of the object 1 can be calculated in advance. 

For this reason, after the expansion coefficient K 
corresponding to the spectral reflectance f(/t) is 
input, the tristimulus values C under the observation 
illumination light corresponding to the spectral 

15 reflectance f(X) can be calculated at a high speed as 

the linear sum of the tristimulus values c( 1 ) of the 
basis functions. 
( Other Embodiments ) 

The present invention is not limited to the above 

2 0 embodiments, and various changes and modifications can 

be made. 

(1) In the first and second embodiments, the 
spectral reflectance f ( X ) of the object 1 is calculated 
from image data obtained by sensing the object 1 with 
2 5 the digital camera 3. However, the present invention 

is not limited to this, and the spectral reflectance 
image data of an arbitrary object can be used. 
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(2) In the first and second embodiments, the CRT 
monitor 10 has been exemplified as the color image 
output means. However, this can be replaced with an 
image output device such as a display of another scheme 

5 or a printer, 

(3) In the first and second embodiments, the 
digital camera 8 with three RGB channels is used as the 
input device for sensing the color chip 6. However, 
for the camera for sensing the color chip 6, an 

10 arbitrary spectral sensitivity characteristic and 

an arbitrary number of channels can be selected. 

(4) In the present invention, a system may be 
constructed in which the sensing side for sensing the 
object 1 is located in a remote site separated from 

15 the observation side for observing an object image 

displayed on the CRT monitor 10, and image data is 
transmitted. 

In this case, the spectral reflectance calculation 
unit 4 or 14, tristimulus value calculation unit 5 or 

20 15, and color image signal calculation unit 9 can be 

provided either on the sensing side or on the 
observation side. 

In this case, the processing described in the 
above embodiments can be realized by storing data 

2 5 necessary for processing by the spectral reflectance 

calculation unit 4 or 14, tristimulus value calculation 
unit 5 or 15, and color image signal calculation unit 9 
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in the respective units or acquiring the data by 
transmission. 

(5) In the first embodiment , the tristimulus 
values C under the observation illumination light of 
5 the object 1 are obtained assuming that the product of 

the spectral reflectance f(X) of the object 1 and the 
color matching functions ci(X) can be expanded by 
combining the spectral reflectance q j ( X ) of the color 
chip 6 and the spectral sensitivity hi( 2 )(/L) of the 

10 digital camera 8 for sensing the color chip 6. Even 

when the spectral reflectance f(X) of the object 1 can 
be expanded by a value obtained by dividing the product 
of the spectral reflectance q j ( X ) of the color chip 6 
and the spectral sensitivity hi( 2 )(/L) of the digital 

15 camera 8 by the color matching functions c±(X), the 

tristimulus values C under the observation illumination 
light of the object can be obtained. 

In this case, when the spectral reflectance f(X) 
of the object 1 is multiplied by a matrix, the spectral 

2 0 reflectance can be converted into the tristimulus 

values C under the observation illumination light. 
By coupling a matrix used to calculate the spectral 
reflectance f(X) from the object image signal obtained 
by sensing the object 1 with the digital camera 3, the 

25 object image signal can be directly converted into the 

tristimulus values C under the observation illumination 
light. 
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The present invention is not limited to the method 
of calculating the tristimulus values under the 
observation illumination light from the spectral 
reflectance of the object and also incorporates a 
5 method of directly calculating the tristimulus values 

under the observation illumination light from the 
signal value obtained by an input device such as 
a camera. 

(6) As a modification to the second embodiment, 

10 the tristimulus values C under the observation 

illumination light can be directly calculated from 
the object image signal by coupling a matrix for 
calculating the expansion coefficient K of the basis 
function V of the spectral reflectance f(>L) of the 

15 object 1 from the object image signal obtained by 

sensing the object to a matrix for calculating the 
tristimulus values C under the observation illumination 
light of the object 1 from the expansion coefficient. 
The present invention also incorporates such 

2 0 a substantially equivalent processing method. 

As has been described above, according to the 
present invention, a color reproduction system having a 
color image processing apparatus for easily converting 
the spectral reflectance image of an object into 

25 a tristimulus image under observation illumination 

light without using any measurement instrument such as 
a spectrophotometer can be provided. 



- 35 - 



Additional advantages and modifications will 
readily occur to those skilled in the art. Therefore, 
the invention in its broader aspects is not limited to 
the specific details and representative embodiments 
5 shown and described herein. Accordingly, various 

modifications may be made without departing from the 
spirit or scope of the general inventive concept as 
defined by the appended claims and their equivalents* 
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WHAT IS CLAIMED IS: 

1. A color reproduction system comprising: 
tristimulus value calculation means for 

calculating tristimulus values under observation 
5 illumination light corresponding to a spectral 

reflectance of an object, said tristimulus value 
calculation means calculating the tristimulus values 
using spectral reflectance data of a color chip formed 
from a plurality of unit color chips , color chip 
10 sensing data obtained by sensing the color chip with an 

input device under the observation illumination light, 
spectral sensitivity data of said input device, and 
color matching function data; 

means for calculating an output color image signal 
15 based on the calculated tristimulus values; and 

means for outputting a color image based on the 
output color image signal. 

2. A system according to claim 1, wherein said 
tristimulus value calculation means multiplies the 

2 0 color chip sensing data by a matrix obtained from 

a relationship between a product of the spectral 
reflectance of the object and color matching functions 
and a product of the spectral reflectance data of the 
color chip and the spectral sensitivity of said input 

25 device, thereby calculating the tristimulus values. 

3. A system according to claim 1, wherein said 
tristimulus value calculation means obtains the 



tristimulus values as a linear sum of basis function 
tristimulus values obtained by multiplying the color 
chip sensing data by a matrix obtained from a 
relationship between a product of a plurality of basis 
functions of the spectral reflectance of the object and 
color matching functions and a product of the spectral 
reflectance data of the color chip and the spectral 
sensitivity of said input device. 

4. A system according to claim 1, wherein each of 
said first image sensing means and said second image 
sensing means comprises a digital camera. 

5. A system according to claim 1, wherein the 
color chip has a plurality of unit color chips having 
independent spectral reflectances and arrayed in a 
matrix. 

6. A system according to claim 1, wherein said 
means for outputting the color image comprises a 
monitor, and said means for outputting the output color 
image calculates the output color image on the basis of 
characteristics of said monitor. 

7. A color reproduction system comprising: 
first image sensing means for sensing an object 

under sensing illumination light; 

means for calculating a spectral reflectance of 
image data of the object sensed by said first image 
sensing means on the basis of spectral sensitivity data 
of said first image sensing means, spectrum data of 
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the sensing illumination light, statistic data of 
a spectral reflectance of the object, and outputting 
spectral reflectance image data corresponding to the 
calculated spectral reflectance; 
5 second image sensing means for sensing a color 

chip under observation illumination light; 

means for calculating tristimulus values of the 
object under the observation illumination light on the 
basis of the output spectral reflectance image data, 

10 color chip image data of the color chip sensed by said 

second image sensing means, spectral sensitivity data 
of said second image sensing means, color chip spectral 
reflectance data representing a spectral reflectance 
distribution of the color chip, and color matching 

15 function data; 

means for calculating an output color image signal 
on the basis of the calculated tristimulus values; and 

means for outputting a color image on the basis of 
the output color image signal. 

20 8. A system according to claim 7, wherein each of 

said first image sensing means and said second image 
sensing means comprises a digital camera, 

9. A system according to claim 7, wherein the 
color chip has a plurality of unit color chips having 

25 independent spectral reflectances and arrayed in 

a matrix. 

10. A system according to claim 7, wherein said 
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means for outputting the color image comprises a 
monitor, and said means for outputting the output color 
image calculates the output color image on the basis of 
characteristics of said monitor. 
5 11. A color reproduction system comprising: 

first image sensing means for sensing an object 
under sensing illumination light; 

means for outputting expansion coefficient data 
which is represented as a linear sum of basis functions 
10 of a spectral reflectance of the object on the basis of 

spectral sensitivity data of said first image sensing 
means, spectrum data of the sensing illumination light 
and statistic data of a spectral reflectance of the 
object; 

15 second image sensing means for sensing a color 

chip under observation illumination light; 

means for calculating tristimulus values of the 
object under the observation illumination light on the 
basis of the output expansion coefficient data, color 

2 0 chip image data of the color chip sensed by said second 

image sensing means, spectral sensitivity data of said 
second image sensing means, color chip spectral 
reflectance data representing a spectral reflectance 
distribution of the color chip, and color matching 

25 function data; 

means for calculating an output color image signal 
on the basis of the calculated tristimulus values; and 
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means for outputting a color image on the basis of 
the output color image signal. 

12. A system according to claim 11, wherein each 
of said first image sensing means and said second image 

5 sensing means comprises a digital camera. 

13. A system according to claim 11, wherein the 
color chip has a plurality of unit color chips having 
independent spectral reflectances and arrayed in a 
matrix. 

10 14. A system according to claim 11, wherein said 

means for outputting the color image comprises a 
monitor, and said means for outputting the output color 
image calculates the output color image on the basis of 
characteristics of said monitor. 
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ABSTRACT OF THE DISCLOSURE 
A color reproduction system includes a tristimulus 
value calculation unit for calculating tristimulus 
values (X, Y, Z) under observation illumination light 
5 corresponding to the spectral reflectance of an object, 

which is obtained by a spectral reflectance calculation 
unit, and a color image signal calculation unit for 
calculating output color image signals (R 1 , G', B') to 
a CRT monitor on the basis of the tristimulus values 

10 (X, Y, Z). The tristimulus value calculation unit 

converts spectral reflectance image data (F) of the 
object into tristimulus value image data (X, Y, Z) 
under the observation illumination light using the 
spectral reflectance data of a color chip formed from 

15 a plurality of unit color chips, color chip image data 

(P') obtained by sensing the color chip with a digital 
camera under the observation illumination light, 
spectral sensitivity data of the digital camera, and 
color matching function data. 
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before that of the application on which priority 
is claimed: 



25/05/1999 



(Day/Month/Year Filed) 
(HUMU!B) 



Priority Not Claimed 

n 



□ 
□ 
□ 
□ 



I hereby claim the benefit under Title 35, United 
States Code, Section 119(e) of any United States 
provisional appl ication (s) listed below; 



(Appl ication No. ) 
{ £H#-t) 



(Filing Date) 
(ftflB) 



I hereby claim the benefit under Title 35, United 
States Code, Section 120 of any United States 
appl ication (s) or 365(c) of any PCT international 
application designating the United States, listed 
below and, insofar as the subject matter of each of 
the claims of this application is not disclosed in 
the prior United States or PCT Information application 
in the manner provided by the first paragraph of 
Title 35, United States Code, Section 112, I 
acknowledge the duty to disclose information 
which is material to patentability as defined 
in Title 37, Code of Federal Regulations, Section 
1. 56 which become available between the filing 
date of the prior application and the national 
or PCT international filing date of application: 



(Status : Patented, Pending, Abandoned) 

(as : mm*im, &m*. ^mm) 



(Status: Patented, Pending, Abandoned) 



I hereby declare that all statements made herein 
of my own knowledge are true and that all 
statements made on information and belief are 
believed to be true; and further that these 
statements were made with the knowledge that 
willful false statements and the like so made 
are punishable by fine or imprisonment, or both, 
under Section 1001 of Title 18 of the United 
States Code and that such willful false statements 
may jeopardize the validity of the application 
or any patent issued thereon. 
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Japanese Language Declaration 

( a # m m m m ) 



POWER OF ATTORNEY: As a named inventor, I 
hereby appoint the following attorney (s) and/br 
agent (s) to prosecute this application and transact 
all business in the Patent and Trademark Office 
connected therewith, (list name and registration 
number) 



John C. Pokoty lo (Reg. No. 36, 242) and Michae 



Straub ( Reg. No. 36, 941 ) 



Send Correspondence to 
STRAUB & P0K0TYL0 



1 Bethany Road 
Suite 56, B I dg . 4 
Haz let, N J 07730 



Direct Telephone Calls to: (name and telephone number) 
Te 1 ephone No. (732)335-1222 





Full name of sole or first inventor 
Kenro Ohsawa 


mftwmnm* an 


Inventor s signature Date 




Residence ^ 
Musash i no-sh i , Japan 


n m 


Citizenship 
JAPAN 


B*mm«My\3E-=FTf?A«iiiKT 2 — 3 


Post Office Address 

c/o Intellectual Property 

Department, 

OLYMPUS OPTICAL CO. , LTD. 




2-3, Kuboyama-cho, HachioJ i -shi t 
Tokyo , Japan 



(Supply similar information and signature for second 
and subsequent joint inventors. ) 



